Exploring Uncertainty with Gaussian Processes: An Interactive Tutorial

1. Introduction to Gaussian processes and why they are useful

Every day we make predictions:

• Will it rain tomorrow?
• What will the temperature be next week?
• How much traffic will I face on the way to work?

Most models give us a single best prediction based on seen data. Linear regression for example fits a single line.

But real life is uncertain: even if a model predicts the temperature tomorrow to be 20 degrees - it could easily be 18°C or 23°C. Thus, we need not just a single prediction, but also a sense of how confident the model is. This is where Gaussian Processes (GPs) are useful. Instead of committing to one best function, a GP represents an infinite number of functions that could explain the data. Some functions are likelier than others, but all are possible.

Once Loop Reflect

In the plot, we visualise the uncertainty around each prediction with darker shaded regions for more confident predictions.

Gaussian Processes (GPs) are a natural extension of well-known regression models and are defined by just two ingredients:

• a mean function: the average trend (often set to zero to “let the data speak”),
• a covariance function (kernel): defining how input points (x-values) move together, i.e. how smooth, noisy the world is.

Why Gaussian Processes are useful

• They provide not only predictions, but also a measure of uncertainty.
• Confidence grows where we’ve seen lots of data, and fades where we haven’t.
• This makes them powerful for guiding decisions, such as where to collect more data or when to be cautious about a forecast.

In this tutorial, we’ll build an intuition for GPs: how to express our beliefs with mean and covariance functions, how GPs make predictions at new points, and how those predictions adapt as new data comes in.